Imaging apparatus and control method thereof

ABSTRACT

The present invention provides an imaging apparatus capable of facilitating a user&#39;s check of a focus adjustment state at the time of multiple exposure shooting, and also allowing a user to easily check in what kind of composition an image to be shot now will be combined with an already captured image immediately before shooting. The imaging apparatus includes a display control unit configured to perform control to display a through-the-lens image captured by an imaging unit on a display unit while sequentially updating the through-the-lens image during a focus adjustment, and display a multiple image combination result image generated by combining the through-the-lens image captured by the imaging unit and at least one already captured image on the display unit while sequentially updating the multiple image combination result image after completion of the focus adjustment.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an imaging apparatus, and, inparticular, to an imaging apparatus capable of combining a plurality ofcaptured images.

2. Description of the Related Art

Conventionally, there has been a technique of realizing multipleexposure shooting by adding a plurality of digital image signals.Japanese Patent Application Laid-Open No. 2003-69888 discusses atechnique related to shooting processing using a multiple exposurefunction of generating a single image through a plurality of exposures.According to the technique discussed in Japanese Patent ApplicationLaid-Open No. 2003-69888, image data of each frame stored in an imagememory is combined with each other by averaging processing, and theresult thereof is stored in the image memory. Japanese PatentApplication Laid-Open No. 2003-125266 discusses a technique fordisplaying a combined image of an already captured image and athrough-the-lens image, and allowing a user to perform multiple exposureshooting after confirming the position where an object is combined.Japanese Patent Application Laid-Open No. 2005-102263 discusses atechnique for combining an image captured by preliminary shooting and athrough-the-lens image to display the combined image, and stopping thedisplay of the combined image upon a start of automatic focus (AF)processing in response to half-pressing of a shutter button to start adisplay showing only the through-the-lens image.

During the multiple exposure shooting as discussed in theabove-described patent documents, Japanese Patent Applications Laid-OpenNo. 2003-69888 and No. 2003-125266, if a displayed image can be switchedfrom a combined image to only a through-the-lens image in response tohalf-pressing of the shutter button as discussed in Japanese PatentApplication Laid-Open No. 2005-102263, it is helpful for a user to checka focus adjustment state at the time of the AF processing. However, inthis case, the user cannot check the composition, i.e., in what kind ofcomposition an image to be shot now will be combined with the alreadycaptured image, after the AF processing is completed and immediatelybefore shooting by fully pressing the shutter button. Therefore, if theuser wants to check the composition of the combined image, the userneeds to release the shutter button first, and then repeat the AFprocessing to shoot an image after adjusting the composition, whichrequires a bothersome operation to the user.

SUMMARY OF THE INVENTION

The present invention is directed to an imaging apparatus capable offacilitating a user's check of a focus adjustment state at the time ofmultiple exposure shooting, and also allowing a user to easily checkwhat kind of composition an image to be shot now will be combined withan already captured image immediately before shooting.

According to an aspect of the present invention, an imaging apparatusincludes an imaging unit, a generation unit configured to generate amultiple image combination result image by combining a through-the-lensimage captured by the imaging unit and at least one already capturedimage, a focus adjustment unit configured to perform a focus adjustmentby driving a focus lens, and a display control unit configured toperform control to display the through-the-lens image captured by theimaging unit on a display unit while sequentially updating thethrough-the-lens image when the focus adjustment unit is performing thefocus adjustment, and display the multiple image combination resultimage generated by the generation unit on the display unit whilesequentially updating the multiple image combination result image afterthe focus adjustment unit completes the focus adjustment.

According to the aspect of the present invention, since the display canbe switched to a more suitable display according to a user's operationstate, it is possible to facilitate a user's check of a focus adjustmentstate at the time of multiple exposure shooting, and also allowing auser to easily check in what kind of composition an image to be shot nowwill be combined with the already captured image immediately beforeshooting.

Further features and aspects of the present invention will becomeapparent from the following detailed description of exemplaryembodiments with reference to the attached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of the specification, illustrate exemplary embodiments, features,and aspects of the invention and, together with the description, serveto explain the principles of the invention.

FIG. 1 is a block diagram illustrating a configuration of a digitalcamera.

FIGS. 2A and 2B illustrate an outer appearance of the digital camera.

FIGS. 3A and 3B illustrate a display example of a menu screen forpreliminary settings regarding multiple exposure shooting.

FIGS. 4A to 4E illustrate image data to be stored in a buffer memory 122during multiple exposure shooting.

FIG. 5 (including FIGS. 5A and 5B) is a flowchart illustrating multipleexposure shooting mode processing.

FIG. 6 is a flowchart illustrating multiple exposure shootingprocessing.

FIGS. 7A and 7B each illustrate a display example in quickreview/playback processing during multiple exposure shooting.

FIG. 8 (including FIGS. 8A and 8B) is a flowchart illustrating multipleexposure shooting mode processing (live view (LV)).

FIG. 9 (including FIGS. 9A and 9B) is a flowchart illustrating multipleimage live view display processing.

FIG. 10A illustrates examples of combination ratios for a multiple imagelive view in an automatic exposure adjustment mode.

FIG. 10B illustrates examples of combination ratios for a simulationlive view in the automatic exposure adjustment mode.

FIGS. 11A to 11D each illustrate a display example in the multiple imagelive view display processing.

FIG. 12 is a flowchart illustrating focus control processing.

DESCRIPTION OF THE EMBODIMENTS

Various exemplary embodiments, features, and aspects of the inventionwill be described in detail below with reference to the drawings.

FIG. 1 is a block diagram illustrating a configuration of a digitalcamera 100 according to an exemplary embodiment of an imaging apparatusto which the present invention can be applied. In FIG. 1, a photographiclens 101 is a detachably attached interchangeable lens constituted by,for example, a zoom lens and a focus lens.

An automatic focus (AF) drive circuit 102 includes, for example, adirect-current (DC) motor or a stepping motor, and functions to focusthe digital camera 100 by changing a position of the focus lens includedin the photographic lens 101 under the control of a microcomputer 123.

A diaphragm drive circuit 104 drives a diaphragm 103 which adjusts alight amount transmitted to an image sensor 112. The microcomputer 123calculates an amount to drive the diaphragm 103, and the diaphragm drivecircuit 104 changes an optical diaphragm value accordingly.

A main mirror 105 is a mirror for selectively guiding a light fluxincident from the photographic lens 101 to a finder side or the imagesensor 112 side. Normally, the main mirror 105 is disposed to reflectthe light flux to guide it to the finder portion. On the other hand,during shooting or a live view display, the main mirror 105 is flippedup and retracts from the light flux to guide the light flux to the imagesensor 112 (mirror-up processing). Further, the main mirror 105 isconfigured as a half mirror having a central portion allowing apart oflight to be transmitted therethrough. Therefore, the main mirror 105allows a part of the light flux to pass therethrough and enter a sensorfor focus detection.

A sub mirror 106 is a mirror for reflecting the light flux transmittedthrough the main mirror 105 to guide it to the sensor for focusdetection (disposed in a focus detection circuit 109).

A mirror drive circuit 107 drives the main mirror 105 under the controlof the microcomputer 123.

A pentagonal prism 108 constitutes the finder. The finder is constitutedby, for example, a focus plate and an eyepiece lens (not illustrated),in addition to the pentagonal prism 108.

The focus detection circuit 109 is a block for focus detection. Afterbeing transmitted through the central portion of the main mirror 105 andreflected by the sub mirror 106, the light flux is incident on thesensor for photoelectrically converting the light flux, which isdisposed in the focus detection circuit 109. A defocus amount to be usedin a focus calculation is determined by calculating an output from thesensor. The microcomputer 123 evaluates the calculation result, andinstructs and causes the AF drive circuit 102 to drive the focus lens.

A shutter drive circuit 111 drives a focal plane shutter 110. How longthe shutter is opened is controlled by the microcomputer 123.

The image sensor 112 is embodied by, for example, a charge coupleddevice (CCD) or a complementary metal-oxide semiconductor (CMOS) sensor,and functions to convert an object image formed by the photographic lens101 into an electric signal.

An analog-to-digital (AD) converter 115 converts an analog output signaloutput from the image sensor 112 into a digital signal.

A video signal processing circuit 116 is realized by a logic device suchas a gate array, and is in charge of various types of video signalprocessing.

A display drive circuit 117 is a drive circuit which causes a displaymember 118 to display an image. The display member 118 is a display unitsuch as a thin film transistor (TFT) liquid crystal display or anorganic electroluminescent (EL) display, and corresponds to a rear sidemonitor of the digital camera 100 in the present exemplary embodiment.

A memory controller 119 stores unprocessed digital image data input fromthe video signal processing circuit 116 into the buffer memory 122, andstores processed digital image data into a recording medium 120.Further, the memory controller 119 reads out the image data from thebuffer memory 122 and the recording medium 120 to output it to the videosignal processing circuit 116. Further, the memory controller 119 canoutput an image stored in the recording medium 120 via an externalinterface 121, which can connect to a computer and the like.

The recording medium 120 is a detachable recording medium such as amemory card. Alternatively, the recording medium 120 may be a recordingmedium integrally mounted in the digital camera 100, or may beconstituted by a plurality of recording media.

The external interface 121 is an interface for connecting the digitalcamera 100 to an external apparatus such as a computer via wired orwireless communication.

The buffer memory 122 is a memory for temporarily storing image data.Various types of images used during multiple exposure shooting are alsostored in the buffer memory 122.

The video signal processing circuit 116 applies filtering processing,color conversion processing, and gamma processing on the digitized imagesignal to generate developed data. In addition, the video signalprocessing circuit 116 applies compression processing such as JointPhotographic Experts Group (JPEG) compression processing on thedeveloped data and then outputs the compressed data to the memorycontroller 119.

The video signal processing circuit 116 can add two or more developeddata pieces stored in the buffer memory 122, and generate high-precisiondata from the developed data by increasing a bit rate for gradation, orsimultaneously perform both the addition of developed data and thegeneration of high-precision data to write the result data back into thebuffer memory 122. Further, the video signal processing circuit 116 canoutput a video signal input from the image sensor 112 and an imagesignal conversely input from the memory controller 119 to the displaymember 118 via the display drive circuit 117. These functions can beswitched according to an instruction from the microcomputer 123.

The video signal processing circuit 116 can output, for example,exposure information and white balance information of a signal of theimage sensor 112 to the microcomputer 123 if necessary. Themicrocomputer 123 issues instructions for white balance control and gainadjustment based on these pieces of information. During continuousshooting processing, shot data is temporarily stored into the buffermemory 122 before any processing is applied thereto, the unprocessedimage data is read out via the memory controller 119, and the videosingle processing circuit 116 applies image processing and compressionprocessing, thereby realizing continuous shooting. The number ofcontinuously shot images varies according to the capacity of the buffermemory 122.

The microcomputer 123 is a main control unit comprehensively controlsthe entire digital camera 100. The microcomputer 123 uses a systemmemory 132 as a work memory, and executes various types of programsrecorded in a non-volatile memory 103.

An operation detection unit 124 detects that an operation member isoperated, and notifies the microcomputer 123 of the state when theoperation member is operated. The microcomputer 123 controls therespective units according to the change of the operation member. Theoperation detection unit 124 can also detect an open and closed state ofa cover 28 (hereinbelow referred to as “card cover 28”) of a slotadapted to contain the recording medium 120 and a battery cover 29.

A switch 1 (125) (hereinbelow referred to as “SW1”) is a switchconfigured to be turned on by half-pressing of a release button 10,which is one of operation members. When the switch SW1 is turned on, themicrocomputer 123 starts shooting preparation processing includingautomatic focus (AF) processing and light metering processing.

A switch 2 (126) (hereinbelow referred to as “SW2”) is a switchconfigured to be turned on by full-pressing of the release button 10,which is one of operation members. When the switch SW2 is turned on, themicrocomputer 123 starts actual shooting processing to capture an imageand record the captured image into the recording medium 120 as an imagefile. Further, when the SW1 and SW2 remain turned on, the continuousshooting is executed.

A liquid crystal drive circuit 127 drives an external liquid crystaldisplay member 128 and a finder internal liquid crystal display member129, which display, for example, a processing status and a message usingcharacters and an image, according to a display content instruction fromthe microcomputer 123. A backlight unit such as a light emitting diode(LED) (not illustrated) is disposed at the finder internal liquidcrystal display member 129, and the LED is also driven by the liquidcrystal drive circuit 127.

The microcomputer 123 can calculate the number of remaining shotsallowed to be shot after checking the remaining capacity of therecording medium 120 via the memory controller 119 based on estimatedvalue data of an image size according to International StandardizationOrganization (ISO) sensitivity, an image size, and an image qualitywhich are set before shooting. The number of remaining sheets allowed tobe shot can be also displayed on the display member 18, the externalliquid crystal display member 128, and the finder internal liquidcrystal display member 129 if necessary.

The non-volatile memory 130 is embodied by, for example, an electricallyerasable programmable read-only memory (EEPROM) or a flash memory, andcan keep data stored even when a power source 131 of the digital camera100 is not turned on. The power source 131 supplies required power tothe respective blocks and drive systems.

FIGS. 2A and 2B illustrate an outer appearance of the digital camera100. FIG. 2A is a perspective view of the digital camera 100 as viewedfrom the front side thereof. FIG. 2B is a perspective view of thedigital camera 100 as viewed from the back side thereof. In theperspective view from the front side, the digital camera 100 isillustrated with the interchangeable photographic lens 101 removedtherefrom.

As illustrated in FIG. 2A, the digital camera 100 includes the releasebutton 10, a main electronic dial 11, an ISO setting button 12, anexposure correction button 13, a shooting mode dial 14, and a diaphragmclosing button 15, as operation members. The diaphragm closing button 15is a button for closing the diaphragm 104 to achieve a set diaphragmstate (F value). By pressing the diaphragm closing button 15 during alive view display in a shooting mode, a photographer can check thebrightness of an image to be captured in the set diaphragm state.

The live view display means a display enabling the display member 118 tofunction as an electronic view finder by causing the display member 118to continuously display an image (a through-the-lens image) captured bythe image sensor 112 substantially in real time in such a state that themain mirror 105 is retracted. During the live view display, an imageformed on the image sensor 112 is converted into a digital signal by theAD converter 115, is developed by the video signal processing circuit116, and then is regenerated as the through-the-lens image. Thegenerated through-the-lens image, or a multiple image combination resultimage generated by combining the through-the-lens image and an alreadycaptured image is displayed on the display member 118. The displayedimage is sequentially updated, and is viewed as a moving image. Forexample, this processing is repeated thirty times per second, therebyrealizing the live view display of 30 frames per second (fps).

The main electronic dial 11 is a rotational operation member, and isused in, for example, an operation such as increasing or reducingvarious types of setting values of shooting conditions or the like,changing a selected item when a photographer selects various items, andswitching images set by set in a playback mode.

As illustrated in FIG. 2B, the digital camera 100 includes aninformation display button 16, a menu button 17, a playback button 18, adeletion button 19, a main switch (SW) 20, and a setting button 21 asoperation members. Further, the digital camera 100 includes asub-electronic button 22, a size enlargement button 23, a size reductionbutton 24, and a multifunction controller 25.

The main SW 20 is an operation member for switching an ON state and anOFF state of the power source of the digital camera 100. Thesub-electronic dial 22 is a rotational operation member, and is usedfor, for example, an operation of changing a selected item when aphotographer selects various items, and an image advancing operation ofswitching a displayed image in the playback mode. A finder eyepieceportion 26 is an eyepiece portion used when a user peers into the finderto view an optical image. A live view button 27 is a button forreceiving an instruction to start the live view display, and aphotographer can turn on and off the live view display by pressing thelive view button 27. The card cover 28 is a cover of a containerconfigured to contain the recording medium 120. The battery cover 29 isa cover of a container configured to contain a battery which functionsas the power source unit 131.

The digital camera 100 can perform multiple exposure shooting by addinga plurality of digital image signals. Multiple exposure shooting isstarted when a user selects “ENABLED” at the item of “MULTIPLE EXPOSURE”shooting to set the multiple exposure shooting mode, on a menu screen,which will be described below with reference to FIG. 3. The digitalcamera 100 adds a plurality of images shot after the multiple exposureshooting mode is set (hereinbelow referred to as “multiple images” or“multiple image combination”) to generate a multiple image combinationresult image, and records the generated image onto the recording medium120 as an image file. The addition processing method for generating themultiple image combination result image recorded in the recording medium120 may be embodied by any one of two methods “additive mode” and“automatic exposure adjustment mode”.

In the additive mode, as indicated by an equation (1), the additionprocessing is performed at a combination ratio, which is calculated byjust adding up luminance of each of a plurality of images to becombined. As a result of the addition as indicated by the equation (1),a recordable maximum value of luminance is set as an upper limit (in asaturated state), even in the case that the luminance may exceed therecordable maximum value of luminance. On the other hand, in theautomatic exposure adjustment mode, as indicated by an equation (2), theaddition processing is performed at such a combination ratio that amultiple image combination result image has a luminance value equal tothe average of the luminance values of the respective images to beadded.

Yn=y1+y2+ . . . +yn−1+yn  (1)

Yn=(y1+y2+ . . . +yn−1+yn)/n  (2)

In these equations:Yn represents the luminance of a multiple image combination image to begenerated by combining “n” pieces of images;y1 represents the luminance of the first image;y2 represents the luminance of the second image;yn−1 represents the luminance of the (n−1)-th image;yn represents the luminance of the n-th image; andn represents the number of images to be added.

The equations (1) and (2) may be respectively converted, and based onequations (3) and (4), which express the same processing as theequations (1) and (2), multiple image combination may be performed withuse of an already combined image of multiple images including the imageimmediately before the n-th image. More specifically, in a case wherethe automatic exposure adjustment function is set to “DISABLED” (theadditive mode), the addition processing is performed as indicated by theequation (3), which will be described below, to obtain the combinationratio as indicated by the above-described equation (1) to performmultiple image combination. On the other hand, in a case where theautomatic exposure adjustment function is set to “ENABLED” (theautomatic exposure adjustment mode), the addition processing isperformed as indicated by the equation (4), which will be describedbelow, to obtain the combination ratio as indicated by theabove-described equation (2) to perform multiple image combination.

Yn=Yn−1+yn  (3)

Yn={(Yn−1)×(n×1)/n}+{yn×1/n}  (4)

In these equations:Yn represents the luminance of a multiple image combination image to begenerated by combining “n” pieces of images;Yn−1 represents the luminance of a multiple image combination resultimage generated by combining (n−1) pieces of images;yn represents the luminance of the n-th image; andn represents the number of images to be added.

When the live view display is started in the multiple exposure shootingmode, the digital camera 100 displays an image generated by combining abase image (in a case where the digital camera 100 is set to use a baseimage), an already captured image that has been already captured afterthe multiple exposure shooting mode had been started, and athrough-the-lens image. The base image means an image selected fromimages recorded in the recording medium 120 before the multiple exposureshooting mode is set as an image to be combined with an image that willbe shot in multiple exposure shooting. As a result, a user can shoot animage while confirming what kind of composition the next shooting imageis combined with other images.

However, in the live view display, if the addition processing isperformed using the same combination ratio as the one used when amultiple image combination result image is generated to be stored in therecording medium 120 as indicated by the above-described equations (1)and (2), a display of a through-the-lens image may have low visibilityto a user (may result in a display of an unclear image to a user). Suchdisplay makes it difficult for a user to adjust the composition todetermine in what kind of composition the user should shoot a next imagewhile viewing the through-the-lens image.

For example, in the additive mode, the addition of luminance at a brightarea may result in saturation of the luminance (the luminance may reachthe maximum luminance), which prevents an easy check of the compositionor the focus state. Whereas in the automatic exposure adjustment mode,luminance values of shot images are averaged. Thus, for example, if thenumber of images that have been already shot in the multiple exposureshooting mode increased, the percentage of the luminance of one imagewill be reduced, and therefore a reduction in the percentage of theluminance of the through-the-lens image will make it difficult to checkthe composition or the focus state of the image.

Therefore, normally, the present exemplary embodiment sets a highercombination ratio for a through-the-lens image than the combinationratio of an already captured image to be combined, when thethrough-the-lens image and the already captured image are combined to bedisplayed in the live view display in the multiple exposure shootingmode. Increasing the combination ratio of the through-the-lens image inthis way can provide a clear display of the through-the-lens imagerelative to the already captured image, and improve the visibility ofthe through-the-lens image. Further, setting a fixed combination ratiofor the through-the-lens image regardless of the number of alreadycaptured images to be combined can further improve the visibility of thethrough-the-lens image.

More specifically, if the multiple image combination is performed forthe live view display, a combination ratio is set according to thefollowing equation (5).

Y1={Yn×(1−M)}+{yt×M}  (5)

In this equation:Y1 represents the luminance of a multiple image combination result imageto be displayed in the live view display;yt represents the luminance of a through-the-lens image;Yn represents the luminance of an image generated by combining “n”pieces of images that have already been captured until that time, i.e.,the luminance of an already captured image to be combined with thethrough-the-lens image; andM represents the combination ratio of the through-the-lens image in themultiple image combination result image to be displayed in the live viewdisplay.

In the above-described equation (5), M is 0.5 or more and less than 1.More specifically, the combination ratio of the luminance of athrough-the-lens image is set to be higher than the total of thecombination ratios of the luminance of already captured images withwhich the through-the-lens image is combined. Alternatively, thecombination ratio of the luminance of a through-the-lens image may beset to be higher than each of the combination ratios of the individualalready captured images to be combined.

The digital camera 100 may be set in such a manner that, in the liveview display, the multiple image combination may be omitted and only athrough-the-lens image is displayed. In this case, the digital camera100 may employ an algorithm for displaying only the through-the-lensimage by substituting 1 for M (M=1) and substituting 0 for thecombination ratio (1−M) of the luminance Yn of the already capturedimage in the equation (5). Displaying only the through-the-lens imagecan facilitate confirmation of the current focus state of thethrough-the-lens image.

Generating a multiple image combination result image to be displayed inthe live view display according to the equation (5) can improve thevisibility of the through-the-lens image. However, the multiple imagecombination result image generated according to the equation (5) has adifferent combination ratio from the multiple image combination resultimage to be generated after the actual shooting according to theequation (1) or (3), or the equation (2) or (4) and recorded in therecording medium 120. Therefore, from the multiple image combinationresult image generated according to the equation (5), it cannot beconfirmed what kind of image is actually generated as a multiple imagecombination result image to be recorded in the recording medium 120(especially, what kind of luminance balance the image will have, and howbright the image will be). Further, the live view display may becomeuseless as reference data for setting shooting conditions such as anexposure.

Therefore, according to the present exemplary embodiment, upon pressingof the diaphragm closing button 15 during the live view display in themultiple exposure shooting mode, the digital camera 100 provides adisplay allowing a user to preliminarily simulate and confirm what kindof image is generated as a multiple image combination result image byactual shooting. This display is referred to as “simulation live viewdisplay”. A multiple image combination result image to be displayed inthe simulation live view display is combined according to an equation(6) in a case where the automatic exposure adjustment function is set to“DISABLED (i.e., in the additive mode)”, and according to an equation(7) in a case where the automatic exposure adjustment function is set to“DISABLED” (i.e., in the automatic exposure adjustment mode) accordingto the setting of the multiple exposure adjustment function.

Ys=Yn+yt  (6)

Ys={Yn×n/(n+1)}+{yt×1/(n+1)}  (7)

In these equations:Ys represents the luminance of a multiple image combination result imageto be displayed in the simulation live view display;yt represents the luminance of a through-the-lens image;Yn represents the luminance of an image generated by combining “n”pieces of images that have been already captured until that time, i.e.,the luminance of an already captured image to be combined with thethrough-the-lens image; andn represents the number of images that have been already acquired untilthat time. In particular, in a case where the digital camera 100 is notset to use a base image, “n” is the number of images shot up to thattime in one set of multiple exposure shooting. Whereas in a case wherethe digital camera 100 is set to use a base image, “n” is the number ofimages shot up to that time in one set of multiple exposure shootingwith the value 1 added thereto.

In the equations (6) and (7), the combination ratio of the luminance ofthe already captured image Yn is the same as the combination ratio inthe multiple image combination result image Yn+1 to be recorded in therecording medium 120 in multiple image combination performed at the timeof actual shooting of one more image.

Multiple image combination based on the above-described equations (1) to(7) may be performed for each color.

Operations performed in multiple exposure shooting for multiple imagecombination at the above-described combination ratio will be describedin detail below.

First, how to set preliminary setting items regarding multiple exposureshooting will be described.

FIGS. 3A and 3B each illustrate a display example of a menu screen wherea user selects the settings of the digital camera 100 regarding multipleexposure shooting. When a user presses the menu button 17 to display amain menu, selects a menu regarding multiple exposure shooting from themain menu, and then finalizes the selection, a menu screen 300 regardingmultiple exposure shooting illustrated in FIG. 3A is displayed on thedisplay member 118.

The menu screen 300 includes menu items 301 to 304. A user can selectarbitrary menu item from the menu items 301 to 304 by operating thesub-electronic dial 22. When the user selects one of the menu items andpresses the setting button 21, a list of setting candidates regardingthe selected menu item is displayed. Then, by selecting a desiredsetting candidate from the displayed setting candidate list by, forexample, operating the sub-electronic dial 22 and pressing the settingbutton 21 again, the user can finalize and set the selected settingcandidate as a setting value.

The menu item 301 is a menu item for selecting whether multiple exposureshooting is performed, and can be set by selecting any of two settingcandidates “ENABLED” and “DISABLED”. Hereinbelow, the setting of thisitem is referred to as “multiple exposure shooting necessary/unnecessarysetting”. The multiple exposure shooting necessary/unnecessary settingis recorded in the system memory 132 or the non-volatile memory 130.When the multiple exposure shooting necessary/unnecessary setting ischanged from “DISABLED” to “ENABLED” according to a user's operation,multiple exposure shooting is started from the next shooting processing.

The multiple exposure shooting necessary/unnecessary setting isautomatically changed from “ENABLED” to “DISABLED” under severalconditions, which will be described below, such as completion ofmultiple exposure shooting upon achievement of the scheduled number ofimages. Even in the middle of the multiple exposure shooting, settingthis item to “DISABLED” according to a user's operation brings an end tothe multiple exposure shooting at that moment. At this time, if it ispossible to generate a file of a multiple image combination resultimage, the digital camera 100 generates the file of the multiple imagecombination result image based on the control of the microcomputer 123.

The menu item 302 is a menu item for selecting the number of images tobe combined in one set of multiple exposure shooting, and can be set byselecting any number of images from the setting candidates 2 to 9.Without a selection of a base image, which will be described below, thenumber of images selected at the menu item 302 is set as the number ofimages scheduled to be shot in multiple exposure shooting. In a casewhere a base image is selected, the number of images selected at themenu item 302 with the value 1 subtracted therefrom is set as the numberof images scheduled to be shot in multiple exposure shooting. The numberof images scheduled to be shot in multiple exposure shooting is recordedin the system memory 132. This item cannot be selected and changed whenmultiple exposure shooting is not completed after one or more image isshot in the multiple exposure shooting. (Hereinbelow, this state isreferred to as “multiple exposure shooting ongoing state”. In thisstate, a multiple exposure shooting ongoing flag is set to 1, as will bedescribed below.)

The menu item 303 is a menu item for selecting whether the automaticexposure adjustment function can be performed in multiple exposureshooting, and can be set by selecting any of two setting candidates“ENABLED” or “DISABLED”. If a user sets the automatic exposureadjustment function to “ENABLED”, the digital camera 100 is set toperform the addition processing according to the above-describedautomatic exposure adjustment mode when generating a multiple imagecombination result image to be recorded in the recording medium 120. Ifa user sets the automatic exposure adjustment function to “DISABLED”,the digital camera 100 is set to perform the addition processingaccording to the above-described additive mode when generating amultiple image combination result image to be recorded in the recordingmedium 120. This item cannot be selected and changed in the multipleexposure shooting ongoing state.

The menu item 304 is a menu item for selecting a base image in multipleexposure shooting, and can be set by selecting one image as a base imagefrom images recorded in the recording medium 120 (images stored beforethe multiple exposure shooting mode is set). This item can be set onlywhen the multiple exposure shooting necessary/unnecessary setting is setto “ENABLED”, and the digital camera 100 is not in the multiple exposureshooting ongoing state. In other words, a user can set a base image atthe menu item 304 only during a period since the multiple exposureshooting necessary/unnecessary setting is set to “ENABLED” until thefirst image is shot.

When the base image is set, the screen illustrated in FIG. 3B isdisplayed. An image 306 is an image set as the base image from theimages recorded in the recording medium 120. The base image is read outfrom the recording medium 120 by the video signal processing circuit 116when one or more image is shot in the multiple exposure shooting mode orwhen a live view display is started, and is loaded onto the buffermemory 122 in a state converted as developed data.

When the base image is selected, a setting value of an image size as ashooting condition (the image size of images to be shot in multipleexposure shooting after that) is set to the same value as the image sizeof the base image. Selecting the base image in this way enablesexecution of multiple exposure shooting with use of a previously shotimage as the first captured image.

According to the present exemplary embodiment, the digital camera 100 isconfigured in such a manner that only an image previously shot by thedigital camera 100 itself can be selected as a base image for thenecessity of maintaining consistency of an image size. However, thedigital camera 100 may be configured in such a manner that even an imageother than images shot by the digital camera 100 can be selected as abase image as long as the image has an image size settable as a shootingcondition in the digital camera 100. Alternatively, the digital camera100 may be configured in such a manner that even an image having animage size that is not settable as a shooting condition in the digitalcamera 100 can be set as a base image after the video signal processingcircuit 116 resizes that image.

The setting of a base image is canceled upon completion of the multipleexposure shooting, and the digital camera 100 is returned to a statewith no base image selected. A cancel image selection button 305 is abutton icon for canceling a selected base image, and selecting andpressing this button brings back the digital camera 100 to a state withno base image selected.

Data stored in the buffer memory 122 during multiple exposure shootingwill be described with reference to FIGS. 4A to 4E. The buffer memory122 stores five types of image data, namely, developed data,high-precision data, multiple image developed data, display multipleimage data, and previous shooting display multiple image data at most,according to shooting.

The developed data is data generated by applying development processingsuch as color processing on an image signal acquired from the imagesensor 112 at the time of immediately preceding shooting. An image fileof each original image to be recorded in the recording medium 120 isgenerated by compressing the developed data according to, for example,the JPEG format.

The high-precision data is image data generated by raising the bit rateof the developed data acquired from the previous shooting for multipleimage combination by the video signal processing circuit 116(hereinbelow referred to as “high-precision processing”), and adding itto high-precision data generated until the previous shooting. Applyingthe high-precision processing can reduce the possibility that gradationmay be saturated according to the multiple image combination processing.The high-precision data may be defined as data generated by applying notonly the high-precision processing but also another processing forfacilitating multiple image combination.

The multiple image developed data is data generated by adding developeddata acquired from the current shooting to the high-precision datagenerated at this time (a combined image of images acquired until theprevious shooting). An image file of a multiple image combination resultimage to be recorded in the recording medium 120 is generated bycompressing the multiple image developed data according to, for example,the JPEG format.

The display multiple image data is data generated by reducing orcompressing the multiple image developed data for display. This data isused in displays in quick review (hereinbelow also referred to as “QR”)and playback processing during the multiple exposure shooting (i.e.,multiple exposure shooting ongoing QR/playback processing) and in QR andplayback processing of the first image during the multiple exposureshooting (i.e., multiple exposure shooting ongoing first imageQR/playback processing), which will be described below. The previousshooting display multiple image data corresponds to the display multipleimage data generated in shooting until the previous shooting. A multipleimage combination and multiple image combination result image may referto an image generated by superimposing, composing, synthesizing, ormerging images by way of example and not of limitation. Multiple imagecombination and multiple image combination result image do not refer tosynthesizing a plurality of images that are placed in a tile-arrangedconfiguration such as is done for a panoramic effect. Multiple imagecombination and multiple image combination result image may be referredto as multiple-synthesize image, multiple-composite image, multipleimage combination image, or multiple-composite image.

FIGS. 4A to 4D illustrates data stored in the buffer memory 122 in eachstate with no base image selected during multiple exposure shooting.

FIG. 4A illustrates the state of the buffer memory 122 after shooting ofa first image. After an image signal A is acquired from the shooting ofthe first image, developed data A generated by developing the imagesignal A is stored in the buffer memory 122. However, at this time, thehigh-precision data, the multiple image developed data, the displaymultiple image data, and the previous shooting display multiple imagedata are neither generated nor stored, and the buffer memory 122 has afree space for storing these types of data. Therefore, it is possible toincrease the speed of other types of processing by assigning, to theother processing, the capacity left for storing the high-precision data,the multiple image developed data, the display multiple image data, andthe previous shooting display multiple image data. Examples of the otherprocessing include face detection processing with use of live viewshooting, and contrast AF processing.

FIG. 4B illustrates the state of the buffer memory 122 after shooting ofa second image. After an image signal B is acquired from the shooting ofthe second image, developed data B is generated by developing the imagesignal B, and is stored in the buffer memory 122. Further, thehigh-precision processing is applied to the developed data A storedafter the shooting of the first image to generate high-precision data A,which is then stored in the buffer memory 122 (the addition is omittedsince no high-precision data is stored after the shooting of the firstimage). The high-precision data A and the developed data B are combinedto generate multiple image developed data A+B, which is then stored inthe buffer memory 122. The multiple image developed data A+B is reducedin size or compressed to generate display multiple image data A+B, whichis then stored in the buffer memory 122. At the time of the shooting ofthe second image, previous shooting display multiple image data isgenerated from the image file of the first image A recorded in therecording medium 120, since no display multiple image data is generatedat the time of the shooting of the first image. Then, the generatedprevious shooting display multiple image data is stored in the buffermemory 122.

FIG. 4C illustrates the state of the buffer memory 122 after shooting ofa third image. After an image signal C is acquired from the shooting ofthe third image, developed data C is generated by developing the imagesignal C, and is then stored in the buffer memory 122. Further, thehigh-precision processing is applied to the developed data B storedafter the shooting of the second image, which is then added to thehigh-precision data A stored after the shooting of the second image. Inthis way, high-precision data A+B is generated and stored in the buffermemory 122. The high precision data A+B and the developed data C arecombined to generate multiple image developed data A+B+C, which is thenstored in the buffer memory 122. The multiple image developed data A+B+Care reduced in size or compressed to generate display multiple imagedata A+B+C, which is then stored in the buffer memory 122. The displaymultiple image data A+B generated at the time of the shooting of thesecond image is stored as previous shooting display multiple image dataA+B.

FIG. 4D illustrates the state of the buffer memory 122 when the imageacquired from the shooting of the third image is discarded by themultiple exposure shooting ongoing QR/playback processing, which will bedescribed below. Discard of the image acquired from the shooting of thethird image results in deletion of the developed data C, the multipleimage developed data A+B+C, and the display multiple image data A+B+Cfrom the buffer memory 122 in the state illustrated in FIG. 4C. Further,the image data stored as the previous shooting display multiple imagedata A+B in the state illustrates in FIG. 4C is stored as the displaymultiple image data A+B in the state illustrated in FIG. 4D.

FIG. 4E illustrates the state of the buffer memory 122 when the thirdimage is shot again after the discard of the shot third image. After animage signal D is acquired from the reshooting of the third image(shooting in the multiple exposure shooting mode after a reshootinginstruction is received), developed data D is generated by developingthe image signal D, and is then stored in the buffer memory 122.Further, the high-precision data A+B, which is stored after the discardof the third image, is kept stored in the buffer memory 122 as it is.The high precision data A+B and the developed data D are combined togenerate multiple image developed data A+B+D, which is then stored inthe buffer memory 122. The multiple image developed data A+B+D isreduced in size or compressed to generate display multiple image dataA+B+D, which is then stored in the buffer memory 122. The displaymultiple image data A+B, which is kept stored after the discard of thethird image, is stored in the buffer memory 122 as previous shootingdisplay multiple image data A+B.

Next, data for use in the multiple exposure shooting processing will bedescribed. The multiple exposure shooting processing is performed withuse of the following variables.

-   -   Multiple exposure shooting necessary/unnecessary setting: This        variable can be set to any of “necessary” or “unnecessary”, and        the setting value thereof is recorded in the non-volatile memory        130 or the system memory 132. If this variable is set to        “necessary”, this indicates that the digital camera 100 is now        in the multiple exposure shooting mode.    -   Multiple exposure shooting ongoing flag: This is a variable        indicating whether multiple exposure shooting is now being        performed, and is recorded in the system memory 132. After the        multiple exposure shooting necessary/unnecessary setting is set        to “necessary”, shooting of one or more image sets the flag to 1        (the multiple exposure shooting ongoing state), and completion        of the multiple exposure shooting sets the flag to 0. The flag        can be used to determine, for example, whether normal processing        or processing during multiple exposure shooting is employed as        processing in a quick review.    -   Number of images scheduled to be shot in multiple exposure        shooting: This is a value indicating the number of times of        multiple exposure shooting performed to generate one multiple        image combination result image (hereinbelow referred to as “one        set of multiple exposure shooting”), and is recorded in the        system memory 132. In a case where the digital camera 100 is not        set to use a base image, the number of images scheduled to be        shot in multiple exposure shooting is equal to the number set at        the menu item 302 illustrated in FIG. 3. In a case where the        digital camera 100 is set to use a base image, the number of        images scheduled to be shot in multiple exposure shooting is        equal to the number set at menu item 302 illustrated in FIG. 3        with the value 1 subtracted therefrom.    -   Number of images completed in multiple exposure shooting: This        is a value indicating the number of images that have been shot        up to that time in one set of multiple exposure shooting, and is        recorded in the system memory 132. When the number of images        completed in multiple exposure shooting reaches the number of        images scheduled to be shot in multiple exposure shooting, this        means that one set of multiple exposure shooting is completed,        and therefore the multiple exposure shooting processing is        ended.    -   Enlargement mode flag: This is a variable for managing the state        whether the digital camera 100 is in an enlargement mode for        enlarging an image in the live view display, and is recorded in        the system memory 132. According to the pressing of the        enlargement button 23, the enlargement mode is sequentially        switched among an enlargement OFF mode (a whole through-the-lens        image is displayed on an entire screen), an enlargement mode        (five times enlargement), and an enlargement mode (ten times        enlargement).

In the multiple exposure shooting ongoing state, information indicatinga storage location on the recording medium 120 of each original imagethat has been shot up to that time in one set of the multiple exposureshooting is recorded in the system memory 132 as already written fileinformation. If there is a plurality of recording media to record animage, information specifying the recording medium of the storagedestination is also recorded therewith.

FIG. 5 (including FIGS. 5A and 5B) is a flowchart illustrating themultiple exposure shooting mode processing. The microcomputer 123develops the program recorded in the non-volatile memory 130 onto thesystem memory 132 to execute it, by which the processing illustrated inFIG. 5 can be realized. When the multiple exposure shootingnecessary/unnecessary setting is set to “necessary”, the processingillustrated in FIG. 5 is started.

In step S500, the microcomputer 123 determines whether the digitalcamera 100 receives an instruction to start the live view display. Morespecifically, the microcomputer 123 determines whether the live viewbutton 27 is pressed. If the microcomputer 123 determines that the liveview button 27 is pressed (the digital camera 100 receives aninstruction for the live view display) (YES in step S500), theprocessing proceeds to step S530, whereas if not (NO in step S500), theprocessing proceeds to step S501. In step S530, the microcomputer 123performs multiple exposure shooting mode processing (LV) while providingthe live view display, which will be described below with reference toFIG. 8.

In step S501, the microcomputer 123 determines whether the SW2 is turnedon. If the SW2 is turned on (YES in step S501), the processing proceedsto step S502, and if the SW2 is not turned on (NO in step S501), theprocessing proceeds to step S507.

In step S502, the microcomputer 123 performs the multiple exposureshooting processing. The details of the multiple exposure shootingprocessing will be described below with reference to FIG. 6.

In step S503, the microcomputer 123 refers to the system memory 132 todetermine whether the number of images completed in multiple exposureshooting completion is 1. In other words, the microcomputer 123determines whether the first image in one set of multiple exposureshooting has been shot in the multiple exposure shooting processingperformed in step S502.

If the number of images completed in multiple exposure shooting is 1(YES in step S503), the processing proceeds to step S504, in which themicrocomputer 123 sets and stores 1 as the multiple exposure shootingongoing flag in the system memory 132. If the microcomputer 123determines in step S503 that the number of images completed in multipleexposure shooting is not 1 (NO in step S503), or if the microcomputer123 sets the multiple exposure shooting ongoing flag to 1 in step S504,the processing proceeds to step S505.

In step S505, the microcomputer 123 determines whether the number ofimages completed in multiple exposure shooting stored in the systemmemory 132 reaches the number of images scheduled to be shot in multipleexposure shooting. If the number of images completed in multipleexposure shooting reaches the number of images scheduled to be shot inmultiple exposure shooting (YES in step S505), this means that one setof multiple exposure shooting is completed. Therefore, the processingproceeds to step S506, in which the microcomputer 123 performs“save-and-exit processing”, and then ends the multiple exposure shootingmode processing.

In the “save-and-exit processing” in step S506, the microcomputer 123ends the multiple exposure shooting at this time, generates a multipleimage combination result image with use of images acquired until thistime, and records the generated image in the recording medium 120 as animage file. At this time, the microcomputer 123 generates the multipleimage combination result image according to the setting of the automaticexposure adjustment function set in the menu screen illustrated in FIG.3. If the automatic exposure adjustment function is set to “ENABLED”,the microcomputer 123 performs the multiple image combination processingbased on the above-described equation (4). If the automatic exposureadjustment function is set to “DISABLED”, the microcomputer 123 performsthe multiple image combination processing based on the above-describedequation (3).

Further, the microcomputer 123 initializes multiple exposure shooting.According to the initialization, the microcomputer 123 discards allimage data pieces recorded in the buffer memory 122. Then, themicrocomputer 123 resets the number of images scheduled to be shot inmultiple exposure shooting and the number of images completed inmultiple exposure shooting, which are recorded in the system memory 132.The microcomputer 123 changes the multiple exposure shootingnecessary/unnecessary setting to “unnecessary”. Further, themicrocomputer 123 sets the multiple exposure shooting ongoing flagstored in the system memory 132 to 0. Then, the microcomputer 123deletes all of the contents of the already written file informationstored in the system memory 132.

In step S507, the microcomputer 123 determines whether the playbackbutton 18 is pressed (whether the digital camera 100 receives aninstruction to shift to the playback mode). If the playback button 18 ispressed (YES in step S507), the processing proceeds to step S508. If theplayback button 18 is not pressed (NO in step S507), the processingproceeds to step S514.

In step S508, the microcomputer 123 refers to the system memory 132 todetermine whether the multiple exposure shooting ongoing flag is 1(whether the digital camera 100 is in the multiple exposure shootingongoing state). If the multiple exposure shooting ongoing flag is 1 (YESin step S508), the processing proceeds to step S510. If the multipleexposure shooting ongoing flag is not 1 (NO in step S508), theprocessing proceeds to step S509.

In step S509, the microcomputer 123 performs normal playback modeprocessing. During the normal playback mode processing, themicrocomputer 123 applies the playback mode processing such as a oneimage display, a multiple image display, image advancing, deletion, andattribute addition to all images that are recorded in the recordingmedium 120 and can be played back by the digital camera 100.

In step S510, the microcomputer 123 determines whether the digitalcamera 100 is set to use a base image. If the digital camera 100 is notset to use a base image (NO in step S510), the processing proceeds tostep S511. If the digital camera is set to use a base image (YES in stepS510), the processing proceeds to step S513.

In step S511, the microcomputer 123 determines whether the number ofimages completed in multiple exposure shooting recorded in the systemmemory 132 is two or more. If the microcomputer 123 determines that thenumber is two or more (YES in step S511), the processing proceeds tostep S513. If the number is less than two (NO in step S511), theprocessing proceeds to step S512. Since the multiple exposure shootingongoing flag is 1 in this case, the fact that the number of imagescompleted in multiple exposure shooting is less than two at this timemeans that the number of images completed in multiple exposure shootingis 1.

In step S512, the microcomputer 123 performs multiple exposure shootingongoing first image QR/playback processing. The multiple exposureshooting ongoing first image QR/playback processing is displayprocessing allowing a user to check the first image shot after thedigital camera 100 starts the multiple exposure shooting mode, and doesnot include a display of an image captured before the digital camera 100starts the multiple exposure shooting mode. During the multiple exposureshooting ongoing first image QR/playback processing, the digital camera100 displays the first image shot after the digital camera 100 startsthe multiple exposure shooting mode on the display member 118.

FIG. 7A illustrates an example of a screen displayed on the displaymember 118 by the multiple exposure shooting ongoing first imageQR/playback processing during multiple exposure shooting (displaycontrol). FIG. 7A illustrates a display example when the digital camera100 is set to combine three images in multiple exposure shooting at theabove-described menu item 302 and to use no base image at theabove-described menu item 304 illustrated in FIG. 3.

A display item 701 includes an icon indicating that the image isacquired by multiple exposure shooting. Further, the display item 701indicates that two more images can be shot to reach the number of imagesscheduled to be shot in multiple exposure shooting. A display item 702is a guide display indicating that an operation selection dialog can bedisplayed by pressing the deletion button 19. An image 703 is an imagewithout any image combined therewith yet, since the digital camera 100has shot only one image until this time.

A user views the displayed image, and starts the next shooting in themultiple exposure shooting if there is no problem in the displayedimage. If the user wants to perform some operation, the user presses thedeletion button 19 to display the operation selection dialog. Theoperation selection dialog may include at least “back to previousimage”, “save and exit”, and “exit without saving” as operation options.

If the user selects “back to previous image”, the user can delete thedisplayed first image, and restart multiple exposure shooting whilemaintaining the current settings regarding multiple exposure shootingwhich are set in the menu screen illustrated in FIG. 3. If the userselects “save and exit”, the digital camera 100 ends the multipleexposure shooting mode at this point. If the user selects “exit withoutsaving”, the digital camera 100 ends the multiple exposure shooting modeat this point without recording the displayed first image into therecording medium 120 (or while deleting it from the recording medium120). When the digital camera 100 performs “save-and-exit processing” or“exit-without-saving processing”, the digital camera 100 sets themultiple exposure shooting ongoing flag recorded in the system memory132 to zero, and sets the multiple exposure shootingnecessary/unnecessary setting to “unnecessary”.

In step S513, the microcomputer 123 performs the multiple exposureshooting ongoing QR/playback processing. The multiple exposure shootingongoing QR/playback processing is display processing allowing a user tocheck images acquired after the digital camera 100 shifts to themultiple exposure shooting mode, and how they are combined by multipleimage combination. The processing does not include a display of an imagecaptured before the digital camera 100 shifts to the multiple exposureshooting mode, other than a base image.

FIG. 7B illustrates an example of a screen displayed on the displaymember 118 by the multiple exposure shooting ongoing QR/playbackprocessing during multiple exposure shooting (display control). As isthe case with the screen illustrated in FIG. 7A, the screen illustratedin FIG. 7B is a display example when the digital camera 100 is set tocombine three images in multiple exposure shooting at theabove-described menu item 302 and to use no base image at theabove-described menu item 304 illustrated in FIG. 3. Further, thisexample is a display example after a second image is shot.

The representation of the display item 704 is the same as the displayitem 701 illustrated in FIG. 7A, and indicates that the number ofremaining images is reduced by one and one more image can be shot toreach the number of images scheduled to be shot in multiple exposureshooting. An image 705 is generated by combining the first image and thesecond image acquired from shooting at this time. In other words, theimage 705 displays the display multiple image data A+B illustrated inFIG. 4B.

The display multiple image data is an image generated by reducing thesize of the multiple image combination result image generated from themultiple image combination of images captured from a start of themultiple exposure shooting mode until then, including a base image ifthe digital camera 100 is set to use the base image, using the equation(3) or (4) according to the mode. More specifically, the multipleexposure shooting is ended at this point and a multiple imagecombination result image is generated by the processing (save-and-exitprocessing) for generating a multiple image combination result image tobe recorded in the recording medium 120 with use of images acquireduntil this time. The display multiple image data is an image generatedfrom the multiple image combination at a same composition ratio as thatof the generated multiple image combination result image.

A user views the displayed image, and starts the next shooting in themultiple exposure shooting if there is no problem in the displayedimage. If the user wants to perform some operation, the user presses thedeletion button 19 to display the operation selection dialog. Theoperation selection dialog may include at least “back to previousimage”, “save and exit”, and “exit without saving” as operation options.

If the user selects “back to previous image”, the user can delete onlyone image shot immediately before, and restart the multiple exposureshooting from the previous image. If the user selects “save and exit”,the digital camera 100 ends the multiple exposure shooting at thispoint, and generates a multiple image combination result image with useof the images acquired until this time to record it in the recordingmedium 120. If the user selects “EXIT WITHOUT SAVING”, the digitalcamera 100 ends the multiple exposure shooting mode at this pointwithout recording the multiple image combination result image generatedwith use of the images acquired until this time in the recording medium120. When the digital camera 100 performs “save-and-exit processing” or“exit-without-saving processing”, the digital camera 100 sets themultiple exposure shooting ongoing flag recorded in the system memory132 to zero, and sets the multiple exposure shootingnecessary/unnecessary setting to “unnecessary”.

In step S14, the microcomputer 123 refers to the system memory 132 todetermine whether the multiple exposure shooting ongoing flag is setto 1. If the multiple exposure shooting ongoing flag is set to 1 (YES instep S514), the processing proceeds to step S515. If the multipleexposure shooting ongoing flag is not set to 1 (NO in step S514), theprocessing proceeds to step S517.

In step S515, the microcomputer 123 determines whether there is anyuncompleted end event. The uncompleted end event is an event causing thedigital camera 100 to terminate the multiple exposure shooting mode inthe middle of the processing, and examples thereof include the followingevents.

Event that changes the multiple exposure shooting necessary/unnecessarysetting to “unnecessary” according to a user's operation

-   -   Event that makes the power source turned off, such as a user's        operation on the main SW 20, opening of the card cover 28,        opening of the battery cover 29, or passage of the auto        power-off time.    -   Event that makes continuation of multiple exposure shooting        impossible according to the conditions of shooting settings. If        there is an uncompleted end event (YES in step S515), the        processing proceeds to step S516. If there is no uncompleted end        event (NO in step S515), the processing proceeds to step S517.

In step S516, the microcomputer 123 performs the “save-and-exitprocessing”. This processing is the same as the above-describedprocessing performed in step S506.

In step S517, the microcomputer 123 determines whether there is anoperation for changing the shooting mode. At this time, the operation tobe determined by the microcomputer 123 is not an operation for directlychanging the multiple exposure shooting necessary/unnecessary setting,but an operation for rotating the shooting mode dial 14. If themicrocomputer 123 determines that the digital camera 100 receives anoperation for changing the shooting mode (YES in step S517), theprocessing proceeds to step S518, whereas if not (NO in step S517), theprocessing proceeds to step S521.

In step S518, the microcomputer 123 determines whether the shooting modeafter the change by the operation for changing the shooting mode is ashooting mode (a first shooting mode) predetermined as a shooting modeallowing continuation of the multiple exposure shooting mode processing.

According to the present exemplary embodiment, the shooting modesallowing continuation of the multiple exposure shooting mode processing,among the shooting modes to which a user can change the mode byoperating the shooting mode dial 14, areas follows: a manual mode; ashutter speed priority mode; a diaphragm priority mode; a programautomatic exposure (AE) mode; and a valve mode.

On the contrary, the shooting modes prohibiting continuation of themultiple exposure shooting mode processing (a second shooting mode),among the shooting modes to which a user can change the mode byoperating the shooting mode dial 14, are as follows: a full automaticmode; a flash emission prohibition mode; a creative full automatic mode;a portrait mode; a scenery mode, a close-up mode, a sports mode, a nightscene portrait mode; and a moving image mode. The reason to classify theshooting modes into the one allowing continuation of multiple exposureshooting and the other in this way will be described below.

If the microcomputer 123 determines in step S518 that the changedshooting mode is the shooting mode allowing continuation of the multipleexposure shooting mode processing (YES in step S518), the processingproceeds to step S519. Whereas if not (NO in step S518), the processingproceeds to step S520.

In step S519, the microcomputer 123 switches the shooting mode accordingto the operation of the shooting mode dial 14, and continues themultiple exposure shooting mode processing. At this time, themicrocomputer 123 maintains the multiple exposure shootingnecessary/unnecessary setting, the multiple exposure shooting ongoingflag, the number of images scheduled to be shot in multiple exposureshooting, the number of images completed in multiple exposure shooting,the already written file information, and the base image (if the digitalcamera 100 is set to use a base image) without changing them.

In step S520, the microcomputer 123 performs the “save-and-exitprocessing”. This processing is the same as the above-describedprocessing performed in step S506. However, if the digital camera 100does not yet have enough images to generate a multiple image combinationresult image, the microcomputer 123 does not generate a multiple imagecombination result image. Due to the execution of the “save-and-exitprocessing” at this time, determination in the subsequent step S521 willbe NO, and therefore the multiple exposure shooting mode processing isautomatically ended according to the operation of the shooting mode dial14. At this time, a screen indicating that the multiple exposureshooting mode processing is automatically ended (the multiple exposureshooting mode is canceled) may be displayed on the display member 118.After the multiple exposure shooting mode is ended, the microcomputer123 switches the shooting mode according to the operation of theshooting mode dial 14.

In step S521, the microcomputer 123 refers to the multiple exposureshooting necessary/unnecessary setting in the system memory 132 or thenon-volatile memory 130, and determines whether multiple exposureshooting is set to “ENABLED”. If multiple exposure shooting is set to“ENABLED” (YES in step S521), the processing returns to step S500, andthe processing is repeated. If multiple exposure shooting is set to“DISABLED” (NO in step S521), the multiple exposure shooting modeprocessing is ended.

Now, a description will be given of the reason to classify the shootingmodes into the one allowing continuation of multiple exposure shootingand the other in the above-described manner.

A multiple image combination result image generated from multipleexposure shooting is not an image faithfully representing a reality,since it is an image generated by adding and combining a plurality ofimages. In other words, multiple exposure shooting is a shooting methodutilized as a kind of method for expressing a photographer's idea ratherthan a shooting method for accurately recording a reality. Therefore, itis desirable to allow a photographer to adjust shooting conditions asmuch as possible during multiple exposure shooting, since in this case,the photographer can have a wider variety of options for expression.Therefore, shooting modes allowing a photographer to adjust moreshooting conditions are classified as the modes allowing continuation ofmultiple exposure shooting.

In other words, the shooting modes allowing continuation of the multipleexposure shooting include more items of the shooting conditions that canbe set to a value specified by a user than the shooting modesprohibiting continuation of the multiple exposure shooting. Further,even in the middle of multiple exposure shooting, the shooting mode canbe switched as long as the mode is classified into the shooting modesallowing continuation of the multiple exposure shooting. Therefore, itis possible to generate a multiple image combination result image bycombining images shot under different shooting modes from each other,and the range of expression can be further extended.

Further, it is especially desirable for the digital camera 100 to beable to adjust luminance of each image to be added in multiple exposureshooting. This is because multiple exposure shooting is characterized inthat a plurality of images is combined by adding the luminance of eachimage, and therefore, the luminance may unintentionally become saturatedbefore long in the additive mode without adjusting the luminance of eachimage. Further, even in another mode than the additive mode, an issuemay be raised without an adjustment of the luminance of each image. Forexample, if an image has excessively high luminance compared to otherimages to be combined, this may result in generation of a multiple imagecombination result image with only that image standing out from theother images. On the contrary, if an image has excessively low luminancecompared to other images to be combined, this may result in generationof a multiple image combination result image with only that image tooinconspicuous. Therefore, shooting modes allowing a user to set at leastone value of an exposure correction, a shutter speed, and a diaphragmvalue, which affect the luminance value of each image, and ISOsensitivity, are classified as the shooting modes allowing continuationof the multiple exposure shooting mode processing.

Further, shooting modes allowing an adjustment of white balance areclassified as the shooting modes allowing continuation of the multipleexposure shooting mode processing, to enable colors to be adjusted basedon comparison among a plurality of images to be combined.

The shooting modes satisfying the above-described requirements as theshooting modes allowing continuation of the multiple exposure shootingmode processing (the first shooting mode) are the manual mode, theshutter speed priority mode, the diaphragm priority mode, the program AEmode, and the valve mode.

In the manual mode, a shutter speed, a diaphragm value, ISO sensitivity,an exposure correction, and white balance can be set to valuesdetermined by a user. In the shutter speed priority mode, a shutterspeed, ISO sensitivity, an exposure correction, and white balance can beset to values determined by a user. In this mode, the diaphragm value isautomatically set by the microcomputer 123 to achieve an appropriateexposure according to the shooting conditions set by a user.

In the diaphragm priority mode, a diaphragm value, ISO sensitivity, anexposure correction, and white balance can be set to values determinedby a user. In this mode, the shutter speed is automatically set by themicrocomputer 123 to achieve an appropriate exposure according to theshooting conditions set by a user. In the program AE mode, ISOsensitivity, an exposure correction, and white balance can be set tovalues determined by a user. In this mode, the shutter speed and thediaphragm value are automatically set by the microcomputer 123 toachieve an appropriate exposure according to the shooting conditions setby a user. In the valve mode, a diaphragm value, ISO sensitivity, whitebalance, and the like can be set to values determined by a user.

On the other hand, the shooting modes prohibiting continuation of themultiple exposure shooting mode processing (the second shooting mode)are shooting modes providing less shooting conditions that a user canspecify than the above-described shooting modes allowing continuation ofthe multiple exposure shooting mode processing. Especially, shootingmodes in which a user cannot adjust at least one value of an exposurecorrection, a shutter speed, and a diaphragm value, which affect theluminance value of each image, ISO sensitivity, and white balance (theyare automatically set) are classified into the shooting modesprohibiting continuation of the multiple exposure shooting modeprocessing.

Examples of these shooting modes include the full automatic mode, theflash emission prohibition mode, the creative full automatic mode, theportrait mode, the scenery mode, the close-up mode, the sports mode, thenight scene portrait mode, and the moving image mode. In these shootingmodes, most of the shooting conditions are automatically set by themicrocomputer 123, and there are few items of the shooting conditionsthat a user can set.

If a user changes the shooting mode to one of these shooting modesduring multiple exposure shooting, the microcomputer 123 automaticallyends the multiple exposure shooting mode processing, accordingly it ispossible to prevent the user from performing multiple exposure shootingin a shooting mode unsuitable to the multiple exposure shooting.Further, for an advanced photographer who knows that these shootingmodes are not suitable to multiple exposure shooting, an operation ofswitching the shooting mode to one of these shooting modes duringmultiple exposure shooting means that the photographer does not want tocontinue the multiple exposure shooting. Therefore, the photographer'stime and effort for ending the multiple exposure shooting mode can besaved by automatically ending the multiple exposure shooting modeaccording to an operation of changing the shooting mode to one of theshooting modes unsuitable to multiple exposure shooting, like thepresent exemplary embodiment.

In addition, these shooting modes, in which many shooting conditions areautomatically set, are often selected by novice users. As describedabove, a multiple image combination result image acquired from multipleexposure shooting is not an image faithfully representing a reality.Therefore, a novice user, who does not understand the mechanism ofmultiple exposure shooting, is likely to misunderstand that alow-quality image is captured or a kind of malfunction occurs. To solvesuch an issue, the present exemplary embodiment prohibits execution ofmultiple exposure shooting in the shooting modes that novice users oftenselect, thereby preventing a user from misunderstanding that a multipleimage combination result image acquired from multiple exposure shootingis a low-quality image or a malfunction occurs.

According to the present exemplary embodiment, a shooting mode isclassified into the shooting mode allowing continuation of multipleexposure shooting or the other in the above-described way, so that auser can perform effective multiple exposure shooting while making fulluse of a wider range of expressions.

FIG. 6 is a flowchart illustrating the above-described multiple exposureshooting processing performed in step S502 illustrated in FIG. 5. Themicrocomputer 123 develops the program recorded in the non-volatilememory 130 onto the system memory 132 and executes it, by which theprocessing illustrated in FIG. 6 can be realized. The shootingprocessing illustrated in FIG. 6 will be referred to as “actualshooting” to distinguish it from shooting processing for acquiring athrough-the-lens image, which will be described below.

In step S601, the microcomputer 123 controls the power source to supplypower if the image sensor 112 and the AD converter 115 are in a powersave mode. Further, the microcomputer 123 controls an exposure. Aftercompletion of the exposure, in step S602, the microcomputer 123 readsout an image signal stored in the image sensor 112, and controls the ADconverter 115 to convert the image signal into a digital signal.

In step S603, the microcomputer 123 instructs the video signalprocessing circuit 116 to generate developed data from the image signalreadout in step S602. As described above with reference to FIG. 4, thegenerated developed data is stored in the buffer memory 122.

In step S604, the microcomputer 123 causes the video signal processingcircuit 116 to compress the developed data generated in step S603. Instep S605, the microcomputer 123 records the compressed data into therecording medium 120 as an image file. This image file is not a combinedimage but an original single image. Subsequently, in step S606, themicrocomputer 123 adds information indicating a storage location of theimage file recorded in step S605 to the already written file informationstored in the system memory 132. Instead of or in addition to theinformation indicating the storage location of the image file, themicrocomputer 123 may add information allowing identification of theimage file (for example, a filename). Further, the microcomputer 123adds 1 to the number of images completed in multiple exposure shooting,which is stored in the system memory 132.

In step S607, the microcomputer 123 determines whether the digitalcamera 100 is set to use a base image. If the digital camera 100 is setto use a base image (YES in step S607), the processing proceeds to stepS608. If the digital camera 100 is not set to use a base image (NO instep S607), the processing proceeds to step S611.

In step S608, the microcomputer 123 refers to the system memory 132 todetermine whether the number of images completed in multiple exposureshooting is two or more. If the number of images completed in multipleexposure shooting is less than two, i.e., images shot up to that time isonly one which has been shot at this time (NO in step S608), theprocessing proceeds to step S609. If the number of images completed inmultiple exposure shooting is two or more (YES in step S608), theprocessing proceeds to step S610.

In step S609, the microcomputer 123 reads out the base image from therecording medium 120 to acquire developed data of the base image, causesthe video signal processing circuit 116 to apply high-precisionprocessing to the developed data of the base image, and stores thehigh-precision data into the buffer memory 122. The process in step S609corresponds to a process treating the base image as the image signal Aacquired from shooting of the first image illustrated in FIG. 4B, andthe image data acquired in step S602 from the shooting at this time asthe image signal B acquired from the shooting of the second imageillustrated in FIG. 4B.

In step S610, the microcomputer 123 causes the video signal processingcircuit 116 to apply high-precision processing to the developed dataacquired from the previous shooting, generates high-precision data byadding the high-precision data generated at this time to thehigh-precision data generated in the previous processing, and stores theresulting high-precision data into the buffer memory 122. In the exampleillustrated in FIGS. 4A to 4E, this process corresponds to, duringshooting of the third image illustrated in FIG. 4C, applyinghigh-precision processing to the developed data B acquired from theshooting of the second image, generating the high-precision data A+B byadding the data B to the high-precision data A generated at the time ofshooting of the second image, and storing the generated high-precisiondata A+B into the buffer memory 122.

On the other hand, in step S611, the microcomputer 123 refers to thesystem memory 132 to determine whether the number of images completed inmultiple exposure shooting is two or more. If the microcomputer 123determines that the number of images completed in multiple exposureshooting is two or more (YES in step S611), the processing proceeds tostep S612. Whereas if the number of images completed in multipleexposure shooting is less than two, i.e., images shot up to that time isonly one which has been shot at this time (NO in step S611), theprocessing proceeds to step S618. When the microcomputer 123 determinesin step S611 that the number of images completed in multiple exposureshooting is only one which has been shot at this time, the buffer memory122 is in the state illustrated in FIG. 4A.

In step S612, the microcomputer 123 determines whether the number ofimages completed in multiple exposure shooting is two. If the number ofimages completed in multiple exposure shooting is not two, i.e., if thenumber of images completed in multiple exposure shooting is three ormore (NO in step S612), the processing proceeds to step S610. If thenumber of images completed in multiple exposure shooting is two (YES instep S612), the processing proceeds to step S613.

In step S613, the microcomputer 123 causes the video signal processingcircuit 116 to apply high-precision processing to the developed dataacquired from the previous shooting to generate high-precision data, andstores the generated high-precision data into the buffer memory 122. Inthe example illustrated in FIGS. 4A to 4E, this process corresponds to,during the shooting of the second image illustrated in FIG. 4B, applyinghigh-precision processing to the developed data A acquired from theshooting of the first image, and storing the generated data into thebuffer memory 122 as the high-precision data A.

In step S614, the microcomputer 123 causes the video signal processingcircuit 116 to combine the high-precision data generated in any of stepsS609, S610, and S613, and the developed data acquired from the shootingat this time, which is generated in step S603. In multiple imagecombination at this time, the images are combined according to thesetting of the automatic exposure adjustment function selected on themenu screen illustrated in FIG. 3 based on the above-described equation(3) or (4). At this time, Yn−1 is the luminance of the high-precisiondata generated in any of steps S609, S610, and S613. Further, yn is theluminance of the developed data acquired from the shooting at this time,which is generated in step S603. The image generated from the multipleimage combination is stored into the buffer memory 122 as a multipleimage combination result image.

In step S615, the microcomputer 123 causes the video signal processingcircuit 116 to reduce or compress the multiple image developed datagenerated in step S614, and stores the compressed data into the buffermemory 122 as display multiple image data.

In step S616, the microcomputer 123 determines whether the digitalcamera 100 is set to perform a quick review (QR). Whether to perform aQR immediately after shooting can be set according to a user's operationin advance. If the digital camera 100 is set to perform a QR (YES instep S616), the processing proceeds to step S617, in which themicrocomputer 123 performs the multiple exposure shooting ongoingQR/playback processing. The multiple exposure shooting ongoingQR/playback processing is the same as the above-described processingperformed in step S513 illustrated in FIG. 5. If the microcomputer 123determines in step S616 that the digital camera 100 is not set toperform a QR (NO in step S616), or after execution of the processing instep S617, the multiple exposure shooting processing is ended. Then, theprocessing proceeds to the above-described step S503 illustrated in FIG.5.

In step S618, the microcomputer 123 determines whether the digitalcamera 100 is set to perform a quick review (QR). If the digital camera100 is set to perform a QR (YES in step S618), the processing proceedsto step S619, in which the microcomputer 123 performs the multipleexposure shooting ongoing first image QR/playback processing. Themultiple exposure shooting ongoing first image QR/playback processing isthe same as the above-described processing performed in step S512illustrated in FIG. 5. If the microcomputer 123 determines in step S618that the digital camera 100 is not set to perform a QR (NO in stepS618), or after execution of the processing in step S619, the multipleexposure shooting processing is ended. Then, the processing proceeds tothe above-described step S503 illustrated in FIG. 5.

FIG. 8 (including FIGS. 8A and 8B) is a flowchart illustrating theabove-described multiple exposure shooting mode processing whileproviding a live view (LV) in step S530 illustrated in FIG. 5. Themicrocomputer 123 develops the program recorded in the non-volatilememory 130 onto the system memory 132 and executes it, by which theprocessing illustrated in FIG. 8 can be realized.

In step S8001, the microcomputer 123 raises the main mirror 105 to guidea light flux of object light to the image sensor 112, i.e., performsmirror-up processing. Further, the microcomputer 123 controls theshutter drive circuit 111 to open the shutter 110.

In step S8002, the microcomputer 123 acquires the multiple imagedeveloped data stored in the buffer memory 122. The multiple imagedeveloped data is a multiple image combination result image generated bycombining images acquired up to that time in one set of multipleexposure shooting (including a base image if the digital camera 100 isset to use a base image).

In step S8003, the microcomputer 123 applies preliminary processing tothe multiple image developed data acquired in step S8002 for multipleimage composition. Examples of the preliminary processing include theabove-described high-precision processing. Further, during the live viewdisplay, a multiple image combination result image is only displayed onthe display member 118 or an external monitor via the external interface121 in a full-screen display mode (the same magnification), andtherefore, the multiple image developed data is resized to the number ofpixels required therefor. As a result, an excessively large processingload may be prevented from being applied to the digital camera 100 dueto processing of an excessively large number of pixels.

In step S8004, if the image sensor 112 and the AD converter 115 are inthe power save mode, the microcomputer 123 controls the power source tosupply power thereto, and initializes the settings for the live viewdisplay.

In step S8005, the microcomputer 123 performs multiple image live viewdisplay processing. The multiple image live view display processing willbe described below with reference to FIG. 9.

In step S8006, the microcomputer 123 determines whether the SW1 isturned on. If the SW1 is turned on (YES in step S8006), the processingproceeds to step S8007. If the SW1 is not turned on (NO in step S8006),the processing proceeds to step S8507. In step S8007, the microcomputer123 performs focus control processing. The focus control processing willbe described below with reference to FIG. 12.

In step S8008, the microcomputer 123 initializes the enlargement modeflag stored in the system memory 132 to an OFF state. As a result, ifthe live view display is performed in the enlargement mode before themultiple exposure shooting processing in step S8007, the enlargementdisplay only showing a through-the-lens image is stopped, and isswitched to the multiple image live view display, which will bedescribed below.

Processing in steps S8503 to S8521 are similar to those in theabove-described steps S503 to S521 illustrated in FIG. 5, so that thedescriptions thereof will be omitted here.

In step S8009, the microcomputer 123 determines whether there is aninstruction to stop the live view display. More specifically, themicrocomputer 123 determines whether the live view button 27 is pressed.If the microcomputer 123 determines that the live view button 27 ispressed (there is an instruction to stop the live view display) (YES instep S8009), the processing proceeds to step S8010. In step S8010, themicrocomputer 123 returns the main mirror 105 to the normal position sothat the main mirror 105 reflects a light flux to guide it to the finderportion (mirror-down processing). Then, the processing proceeds to stepS500 in FIG. 5. If the live view button 27 is not pressed (NO in stepS8009), the processing proceeds to step S8011.

In step S8011, the microcomputer 123 determines whether a manual focus(MF) operation is performed. The manual focus operation is, for example,an operation of rotating a focus ring disposed around a lens barrel ofthe photographic lens 101, by which the focus lens is driven (a focusadjustment operation). The manual focus operation may be not only theoperation of the focus ring but also any another operation for a manualfocus adjustment. If the microcomputer 123 determines that the MFoperation is performed (YES in step S8011), the processing proceeds tostep S8012. If the microcomputer 123 determines that the MF operation isnot performed (NO in step S8011), the processing returns to step S8005to repeat the processing.

In step S8012, the microcomputer 123 performs the live view displayshowing only a through-the-lens image on the display member 118 or theexternal monitor without combining the through-the-lens image with themultiple image developed data preprocessed in the above described stepS8003 illustrated in FIG. 8. However, indications of various types ofinformation and icons may be displayed by being superimposed on theimage. Alternatively, the microcomputer 123 may display an imagegenerated by combining the through-the-lens image and the alreadycaptured multiple image developed data at the combination ratio 100:0,which substantially produces the same result as the above-describedprocessing. FIG. 11B illustrates a display example at this time.

If the digital camera 100 performs the live view display showing acombined image of the already captured multiple image developed data andthe through-the-lens image before step S8012, the microcomputer 123 maygradually reduce the combination ratio of the multiple image developeddata to smoothly change the display to a display showing only thethrough-the-lens image. Stopping a multiple image display and starting adisplay showing only the through-the-lens image in this way facilitate auser's determination whether the through-the-lens image is in focusduring the manual focus operation, thereby facilitating the manual focusoperation.

In step S8013, the microcomputer 123 determines whether a predeterminedtime has passed since completion of the MF operation. The predeterminedtime may be any time which is long enough to assume that the user's MFoperation and check of the through-the-lens image along therewith havebeen completed. For example, the predetermined time can be severalseconds. If the microcomputer 123 determines that the predetermined timehas not passed yet (NO in step S8013), the microcomputer 123 waits untilthe predetermined time has passed from completion of the MF operation.If the microcomputer 123 determines that the predetermined time haspassed (YES in step S8013), the processing proceeds to step S8014.

In step S8014, the microcomputer 123 returns the combination ratio inthe live view to the combination ratio before step S8012. Accordingly,if the digital camera 100 is not in the enlargement mode, during themultiple image live view display processing in step S8005, the live viewdisplay is performed to show a multiple image combination result imagegenerated by combining the multiple image developed data preprocessed instep S8003 and the through-the-lens image, as will be described belowwith reference to FIG. 9. After completion of the processing in stepS8014, the processing returns to step S8005 to repeat the processing.

FIG. 9 (including FIGS. 9A and 9B) is a flowchart illustrating theabove-described multiple image live view display processing performed instep S8005 illustrated in FIG. 8. The microcomputer 123 develops theprogram recorded in the non-volatile memory 130 onto the system memory132 to execute it, by which the processing illustrated in FIG. 9 can berealized. FIG. 11A illustrates an example of the multiple imagedeveloped data. This image is an image generated from multiple imagecombination of images captured up to that time, and corresponds to themultiple image developed data acquired in step S8002. FIG. 11Billustrates the display example of only a through-the-lens image. In themultiple image live view display processing, the digital camera 100displays, as the live view display, an image generated from multipleimage combination of the multiple image developed data illustrated inFIG. 11A and the through-the-lens image illustrated in FIG. 11B at adifferent combination ratio according to whether this is a simulationlive view.

In step S901, the microcomputer 123 determines whether the enlargementbutton 23 is pressed. If the microcomputer 123 determines that theenlargement button 23 is pressed (YES in step S901), the processingproceeds to step S902. If the enlargement button 23 is not pressed (NOin step S901), the processing proceeds to step S903.

In step S902, the microcomputer 123 changes the mode according to thepressing of the enlargement button 23. More specifically, when theenlargement button 23 is pressed in step S901, the microcomputer 123refers to the enlargement mode flag stored in the system memory 132.Then, if the enlargement mode flag currently indicates the enlargementOFF mode (enlargement is not effective), the microcomputer 123 changesthe mode to the enlargement mode (five times enlargement). If theenlargement mode flag currently indicates the enlargement mode (fivetimes enlargement), the microcomputer 123 changes the mode to theenlargement mode (ten times enlargement). Further, if the enlargementmode flag currently indicates the enlargement mode (ten timesenlargement), the microcomputer 123 changes the mode to the enlargementOFF mode.

If the microcomputer 123 changes the mode to the enlargement mode (fivetimes enlargement), the microcomputer 123 stops the multiple imagedisplay in the live view display, and starts a display showing only athrough-the-lens image while enlarging the full screen display thereof(a display of the through-the-lens image at a maximum size allowing thewhole image to be contained in the entire display area) five times. Ifthe microcomputer 123 changes the mode to the enlargement mode (tentimes enlargement), the microcomputer 123 starts a display showing onlythe through-the-lens image while enlarging the full screen displaythereof ten times. If the microcomputer 123 changes the mode to theenlargement OFF mode (enlargement is not effective), the microcomputer123 restarts the multiple image display of the through-the-lens imageand the already captured image (the multiple image developed datapreprocessed in the above-described step S8003). The microcomputer 123records the information about the enlargement mode as a result of thechange into the system memory 132 as the enlargement mode flag.

In step S903, the microcomputer 123 refers to the enlargement mode flagstored in the system memory 132 to determine whether the enlargementmode flag currently indicates the enlargement mode. If the enlargementmode flag indicates the enlargement mode (five times enlargement) or theenlargement mode (ten times enlargement), the microcomputer 123determines that the digital camera 100 is currently in the enlargementmode (YES in step S903), and then the processing proceeds to step S920.If the enlargement mode flag indicates enlargement OFF mode (enlargementis not effective), the microcomputer 123 determines that the digitalcamera 100 is not currently in the enlargement mode (NO in step S903),and then the processing proceeds to step S904.

In step S904, the microcomputer 123 determines whether the currentdisplay type is the live view display showing only a through-the-lensimage. The display type during the live view can be switched between adisplay showing multiple images and the live view display showing only athrough-the-lens image without showing multiple images, according topressing of the information display button 16 (corresponding to theoperation in step S927, which will be described below). If themicrocomputer 123 determines that the current display type is the liveview display showing only a through-the-lens image (YES in step S904),the processing proceeds to step S920. Whereas if not (NO in step S904),the processing proceeds to step 905.

In step S905, the microcomputer 123 determines whether the diaphragmclosing button 15 is pressed. If the microcomputer 123 determines thatthe diaphragm closing button 15 is pressed (currently being pressed)(YES in step S905), the processing proceeds to step S912. If themicrocomputer 123 determines that the diaphragm closing button 15 is notpressed (i.e., released) (NO in step S905), the processing proceeds tostep S906.

In step S906, the microcomputer 123 controls the image sensor 112 andthe diaphragm drive circuit 104 to perform a normal exposure foracquisition of a through-the-lens image to thereby acquire thethrough-the-lens image. During the normal exposure for acquisition of athrough-the-lens image, the diaphragm 103 is opened (the diaphragm valueis minimized) to reduce the depth of field to facilitate a check of thefocus state in the live view display, regardless of a diaphragm valueset for actual shooting with use of the SW2. Then, an exposure isperformed while adjusting the charge accumulation time and sensitivity(ISO sensitivity) of the image sensor 112 to achieve brightness suitablefor viewing.

In step S907, the microcomputer 123 applies the development processingto the through-the-lens image acquired from the exposure in step S906.

In step S908, the microcomputer 123 combines the through-the-lens imagedeveloped in step S907 and the multiple image developed datapreprocessed in the above-described step S803 in FIG. 8 at a combinationratio for the multiple image live view. The combination ratio for themultiple image live view is the combination ratio indicated by theabove-described equation (5). In other words, the combination ratio forthe multiple image live view is such a combination ratio that thecombination ratio of a through-the-lens image is set to M (M is 0.5 ormore), and the combination ratio of multiple image developed data is setto (1−M). As a result, during the multiple image live view displayshowing the image generated from multiple image combination of thethrough-the-lens image and the already captured image, a user can moreeasily view the through-the-lens image that will be shot now compared tothe already captured image (multiple image developed data).

FIG. 10A illustrates examples of the combination ratio for the multipleimage live view. For example, as the combination ratio for the multipleimage live view in the automatic exposure adjustment mode, thecombination ratio of a through-the-lens image is fixed to 60% (M=0.6),and the combination ratio of multiple image developed data, which is analready captured image, is set to 40% (1−M=0.4). If two or more imageshave been already shot in one set of multiple exposure shooting (or if abase image is set to be used, and one or more image has been alreadyshot), the multiple image developed data is in such a state that severalimages have been already evenly combined. Therefore, the combinationratio per image for the already captured images according to thecombination ratio for the multiple image live view is calculated byevenly dividing 40%, which is the combination ratio of the multipleimage developed data, by the number of the already captured images.

Due to the employment of this combination ratio, the multiple image liveview display shows the already captured images lightly while showing thethrough-the-lens image relatively clearly. Therefore, a photograph caneasily adjust the composition in consideration of how an image to beshot will be combined with the already captured images. In addition, thephotographer can easily bring the digital camera 100 into focus andadjust the shutter timing for an image to be shot.

In step S909, the microcomputer 123 resizes a multiple image combinationresult image generated by combining the through-the-lens image and thealready captured image (multiple image developed data) in step S908,according to an output destination (the display member 118 or theexternal monitor). For example, if the output destination is the displaymember 118 of the camera main body, the microcomputer 123 resizes theimage into an image with a width of 720 pixels and a height of 480pixels. If the output destination is the external monitor which is ahigh vision monitor, the microcomputer 123 resizes the image into animage with a width of 1620 pixels and a height of 1080 pixels. Further,the microcomputer 123 may resize the image after trimming the upper andlower portions thereof, into an image with a width of 1920 pixels and aheight of 1080 pixels.

In step S910, the microcomputer 123 displays the multiple imagecombination result image resized in step S909 on the display member 118or the external monitor (the multiple image live view display). FIG. 11Cillustrates a display example of the multiple image live view display.In the example illustrated in FIG. 11C, the already captured image isdisplayed lightly, while the through-the-lens image is displayedrelatively clearly as a result of the combination processing in theabove-described step S908, so that a user can easily check thethrough-the-lens image especially. In step S911, the microcomputer 123calculates an exposure amount to be used in acquisition of athrough-the-lens image (image capturing) for the next frame based on thethrough-the-lens image acquired in step S906, sets the diaphragm valueto an open value, and sets the charge accumulation time and thesensitivity of the image sensor 112.

On the other hand, if the microcomputer 123 determines in step S905 thatthe diaphragm closing button 15 is currently being pressed (YES in stepS905), the processing proceeds to step S912 to start a simulationdisplay of a multiple image combination result image that will beactually generated from actual shooting.

In step S912, the microcomputer 123 drives the diaphragm 103 accordingto the diaphragm value (the diaphragm value set by the user or thediaphragm value automatically calculated based on a program diagram) asa shooting condition set for actual shooting with use of the SW2. Themicrocomputer 123 can set the same depth of field for thethrough-the-lens image as the depth of field of an image to be shot inactual shooting by driving the diaphragm 103 to the diaphragm value setfor actual shooting, and allow a user to check whether the image blurs.

In step S913, the microcomputer 123 fixes the diaphragm value set foractual shooting and adjusts the charge accumulation time and thesensitivity to achieve brightness closer to the brightness of the imagethat will be acquired from actual shooting under the current shootingconditions set for actual shooting (simulation adjustment). First, themicrocomputer 123 sets the charge accumulation time within the range ofcharge accumulation time for acquisition of a through-the-lens image toapproach the shutter speed set as one of the shooting conditions. Then,the microcomputer 123 adjusts the sensitivity and supplement a shortfallfrom the adjustment of the charge accumulation time to achieve the setshutter speed.

For example, in a case where a through-the-lens image is updated every1/30 second, the charge accumulation time for acquisition of athrough-the-lens image cannot be set to be longer than 1/30 second.Therefore, in a case where the shutter speed is set to be longer than1/30 second as the shooting condition, the microcomputer 123 sets thecharge accumulation time to the settable longest time for acquisition ofa through-the-lens image, and amplifies the sensitivity by an amountcorresponding to the shortfall to achieve the shutter speed.

In step S914, the microcomputer 123 acquires a through-the-lens image byperforming an exposure according to the diaphragm value, the chargeaccumulation time, and the sensitivity set in steps S912 and S913(simulation exposure).

In step S915, the microcomputer 123 applies the development processingto the through-the-lens image acquired by the exposure in step S914.

In step S916, the microcomputer 123 combines the through-the-lens imagedeveloped in step S915 and the multiple image developed datapreprocessed in the above-described step S8003 in FIG. 8 at acombination ratio for the simulation live view. The combination ratiofor the simulation live view is the combination ratio indicated by theabove-described equations (6) or (7). A user can check what kind ofimage will be generated as a multiple image combination result imagefrom actual shooting under the current shooting conditions beforeexecution of the actual shooting operation by viewing the simulationlive view display.

FIG. 10B illustrates examples of the combination ratio for thesimulation live view. The combination ratio for the simulation live viewin the automatic exposure adjustment mode is determined so that athrough-the-lens image and an already captured image are combined at thesame combination ratio. In other words, the combination ratio of athrough-the-lens image varies according to the number of alreadycaptured images.

For example, in a case where the digital camera 100 is not set to use abase image, and has shot three images up to that time in one set ofmultiple exposure shooting, the combination ratio of thethrough-the-lens image is 1/4 (25%), and the combination ratio of themultiple image developed data is 3/4 (75% in total; 25% for each image).Due to this display, a photographer can check what kind of image will begenerated as a multiple image combination result image in terms ofcombination balance, brightness, density, and a depth of field fromactual shooting under the current shooting conditions, and thereby canappropriately adjust the shooting conditions.

In step S917, the microcomputer 123 resizes the multiple imagecombination result image generated by combining the through-the-lensimage and the already captured image (multiple image developed data) instep S916 according to an output destination (the display member 118 orthe external monitor).

In step S918, the microcomputer 123 displays the multiple imagecombination result image resized in step S917 on the display member 118or the external monitor (the simulation live view display). FIG. 11Dillustrates a display example of the simulation live view display. Theimage displayed in FIG. 11D is an image reflecting the shootingconditions and the setting state of the automatic exposure adjustmentfunction, and therefore does not necessarily provide clear visibility ofthe through-the-lens image. However, this image enables a user to knowhow a multiple image combination result image generated from actualshooting will look like.

Further, since the image illustrated in FIG. 11D is an image reflectingthe shooting conditions, a dialog 1101 is displayed to notify a user ofthe currently set shooting conditions. The dialog 1101 illustrated inFIG. 11D indicates a shutter speed (time value (Tv)), a diaphragm value(aperture value (Av)), an exposure correction, a number of remainingimages that can be shot, ISO sensitivity, and a remaining batterycapacity from the left side in this order. A user can change theshooting conditions while confirming what kind of effect the change inthe shooting conditions provides by viewing the dialog 1101 and thesimulation live view display.

In step S919, the microcomputer 123 calculates an exposure amount to beused in acquisition (image capturing) of a through-the-lens image forthe next frame based on the through-the-lens image acquired in stepS914, sets the diaphragm value to the value set as the shootingcondition, and sets the charge accumulation time and the sensitivity ofthe image sensor 112.

The processes insteps S920, S921, and S922 performed in the enlargementmode or the live view mode displaying only a through-the-lens image aresimilar to the processes in steps S906, S907, and S909, respectively,and therefore the descriptions thereof will be omitted here.

In step S923, the microcomputer 123 displays only the through-the-lensimage resized in step S922 on the display member 118 or the externalmonitor without performing multiple image combination. However,indications of various types of information and icons may be displayedby being superimposed on the image.

FIG. 11B illustrates the display example at this time. The process instep S924 is similar to the process in step S911, and therefore thedescription thereof will be omitted here.

In this way, in the enlargement mode or the live view mode displayingonly a through-the-lens image, only a through-the-lens image isdisplayed in the live view without being combined with another image.Instead, the processes in steps S906 to S911 may be performed at thecombination ratio of the through-the-lens image and the already capturedmultiple image developed data set to 100:0, which substantially producesthe same result as the above-described processes.

In step S925, the microcomputer 123 determines whether the user performsan operation for changing the shooting conditions. If the user performsan operation for changing the shooting conditions (YES in step S925),then in step S926, the microcomputer 123 changes the shooting conditionsaccording to the operation (setting of shooting conditions). Theshooting conditions changeable according to a user's operation includeat least a shutter speed (Tv value), a diaphragm value (Av value), anexposure correction, ISO sensitivity, white balance, and others. Amongthem, the shooting conditions changeable according to a user's operationvary according to the shooting mode set by the shooting mode dial 14 asdescribed above.

In step S927, the microcomputer 123 determines whether the user performsan operation for changing the display type by pressing the informationdisplay button 16. If the microcomputer 123 determines that theinformation display button 16 is pressed (YES in step S927), then instep S928, the microcomputer 123 changes the display type. If thecurrent display type is the live view mode displaying multiple images,the microcomputer 123 changes the display mode to the live view modedisplaying only a through-the-lens image. Further if the current displaytype is the live view mode displaying only a through-the-lens image, themicrocomputer 123 changes the display mode to the live view modedisplaying multiple images.

FIG. 11B illustrates the display example of the live view modedisplaying only a through-the-lens image. The image illustrated in FIG.11B shows only a through-the-lens image, and a user can easily performan operation for affecting only the through-the-lens image such as acheck of a focus state or an adjustment of shutter timing by viewing theimage. The digital camera 100 may switch the display mode from the liveview mode displaying multiple images to the live view mode displayingonly a through-the-lens image while gradually changing the combinationratio to smoothly change the display, instead of changing the displayinstantly. If the microcomputer 123 determines in step S927 that theuser does not perform an operation for changing the display type (NO instep S927), or after the microcomputer 123 completes the process in stepS928, the multiple image live view display processing is ended. Then,the processing proceeds to step S8006 illustrated in FIG. 8.

FIG. 12 is a flowchart illustrating the above-described focus controlprocessing performed in step S8007 in FIG. 8. The microcomputer 123develops the program recorded in the non-volatile memory 130 onto thesystem memory 132 to execute it, by which the processing illustrated inFIG. 12 can be realized.

In step S1201, the microcomputer 123 performs the live view displayshowing only a through-the-lens image on the display member 118 or theexternal monitor without combining the image with the multiple imagedeveloped data preprocessed in the above-described step S8003 in FIG. 8.However, indications of various types of information and icons may bedisplayed by being superimposed the image. Alternatively, themicrocomputer 123 may display an image generated by combining thethrough-the-lens image and the already captured multiple image developeddata at the combination ratio 100:0, which substantially produces thesame result as the above-described processing. FIG. 11B illustrates thedisplay example at this time. In a case where the display mode beforestep S1201 is the live view mode displaying a combined image of thealready captured multiple image developed data and the through-the-lensimage, the microcomputer 123 may gradually reduce the combination ratioof the multiple image developed data to smoothly change the display to adisplay only showing the through-the-lens image.

In step S1202, the microcomputer 123 starts shooting preparationprocessing. The shooting preparation processing includes at least theautomatic focus processing (focus adjustment processing) and the lightmetering processing (or automatic exposure (AE) processing).

In step S1203, the microcomputer 123 determines whether the automaticfocus (AF) processing is completed. If the microcomputer 123 determinesthat the AF processing is not yet completed (NO in step S1203), theprocessing proceeds to step S1204. If the microcomputer 123 determinesthat the AF processing is completed (YES in step S1203), the processingproceeds to step S1205.

In step S1204, the microcomputer 123 determines whether the SW1 isturned OFF, i.e., whether half-pressing of the release button 10 isreleased. If the SW1 is not turned OFF (NO in step S1204), theprocessing returns to step S1203, in which the microcomputer 123continues the shooting preparation processing. If the SW1 is turned OFF(YES in step S1204), the processing proceeds to step S1209.

In step S1205, the microcomputer 123 returns the combination ratio ofthe live view to the state before the execution of the process in stepS1201. As a result, in a case where the digital camera 100 is not in theenlargement mode, the live view display is performed, showing themultiple image combination result image generated by combining themultiple image developed data preprocessed in step S8003 and thethrough-the-lens image in the multiple image live view displayprocessing in step S8005. In other words, if the digital camera 100 isnot in the enlargement mode, the microcomputer 123 causes the multipleimage live view display or the simulation live view display describedabove with reference to FIG. 9.

In step S1206, the microcomputer 123 determines whether the SW2 isturned ON. If the SW2 is not turned ON (NO in step S1206), theprocessing proceeds to step S1207. If the SW2 is turned ON (YES in stepS1206), the processing proceeds to step S1208.

In step S1207, the microcomputer 123 determines whether the SW1 isturned OFF, i.e., whether half pressing of the release button 10 isreleased. If the SW1 is not turned OFF (NO in step S1207), theprocessing returns to step S1206. If the SW1 is turned OFF (YES in stepS1207), the processing proceeds to step S1209.

In step S1208, the microcomputer 123 performs the multiple exposureshooting processing. The multiple exposure shooting processing isalready described with reference to FIG. 6, and therefore thedescription thereof will be omitted here. After completion of theprocess in step S1208, the focus control processing is ended, and theprocessing proceeds to step S8008 illustrated in FIG. 8.

On the other hand, in step S1209, since the SW1 is turned OFF beforeexecution of actual shooting, the microcomputer 123 returns thecombination ratio of the live view to the state before the execution ofthe process in step S1201. Then, the processing proceeds to step S8507illustrated in FIG. 8.

According to the processing illustrated in FIG. 12, when a user pressesthe SW1 to start the AF processing, the microcomputer 123 stops the liveview display showing multiple images, and starts the display showingonly the through-the-lens image. Upon completion of the AF processing,the microcomputer 123 returns the display to the live view displayshowing multiple images. Accordingly, during the AF processing, the usercan easily check the focus state through the AF processing by viewingthe through-the-lens image. Further, after the AF processing iscompleted and the image is set in focus, the microcomputer 123 restartsthe live view display showing multiple images, so the user can make afinal adjustment of the composition in preparation for actual shootingwith use of the SW2.

The microcomputer 123 may control the live view display to show only thethrough-the-lens image for a predetermined time from completion of theAF processing, and return the display to the live view display showingmultiple images after the predetermined time has passed from thecompletion of the AF processing, so that the user can more closely checkthe result of the AF processing. In this case, the processingillustrated in FIG. 12 may add a step for determining whether thepredetermined time (several seconds) has passed after the microcomputer123 determines that the AF processing is completed in step S1203, toimmediately before step S1205, and the processing may proceed to stepS1205 if it is determined that the predetermined time has passed.

As described above, according to the present exemplary embodiment of thepresent invention, the microcomputer 123 sets the combination ratio ofthe through-the-lens image to be higher than the combination ratio ofthe already captured image to be combined, during the live view displayshowing the through-the-lens image and the already captured image whilecombining them. Accordingly, the through-the-lens image can be clearlydisplayed, and the visibility of the through-the-lens image can beenhanced. Moreover, the visibility of the through-the-lens image can befurther enhanced by setting the combination ratio of thethrough-the-lens image to a fixed value regardless of the number ofalready captured images to be combined.

On the other hand, according to the present exemplary embodiment of thepresent invention, it is possible to change the combination ratio of thethrough-the-lens image and the already captured image to be combinedaccording to a user's instruction, to allow a user to preliminarytentatively check what kind of image will be generated as a multipleimage combination result image to be generated from actual shooting andrecorded to the recording medium 120. In this way, it is possible toperform the live view display with the through-the-lens image combinedat a more suitable combination ratio according to a user's intentionduring multiple exposure shooting.

Further, it is possible to facilitate a user to check an adjusted focusstate by stopping the multiple image live view display and starting thedisplay showing only the through-the-lens image during the focusadjustment by the AF or MF processing. Then, since the microcomputer 123returns the display to the multiple image live view display aftercompletion of the focus adjustment, the user can shoot an image afterconfirming and adjusting the composition as to how an image to be shotnow will be combined with the already captured image immediately beforea shooting instruction (SW2).

The above exemplary embodiment is described based on an example in whichmultiple image combination is performed with use of developed data.However, multiple image combination may be performed with use of rawimage data before the development.

Further, according to the above-described exemplary embodiment, thecombination ratio is switched between the combination ratio for themultiple image live view (a first combination ratio) and the combinationratio for the simulation live view (a second combination ratio)according to whether the diaphragm closing button 15 is pressed.However, the combination ratio may be switched among a wider variationof combination ratios according to a user's purpose.

For example, one possible method is to further increase the combinationratio of the through-the-lens image, for example, in a shooting modesuch as the night scene shooting mode that may show the through-the-lensimage extremely darkly, or when it is detected that the acquiredthrough-the-lens image is dark. Further, in the above-describedexemplary embodiment, the simulation live view display is performed aslong as the diaphragm closing button 15 is pressed. However, the presentinvention is not limited to this configuration as long as the displaycan be switched between the simulation live view display and themultiple image live view display according to a user's operation.

Further, the control of the microcomputer 123 may be realized by asingle hardware device, or a plurality of hardware devices may worktogether to control the entire apparatus while appropriately dividingthe processing among them.

Further, the above exemplary embodiment has been described based on anexample in which the present invention is applied to a digital camera,but the present invention is not limited to this example. The presentinvention can be applied to any imaging apparatus including an imagingunit. In other words, the present invention can be applied to a digitalcamera, a digital video camera, a personal computer or personal digitalassistant (PDA) equipped with a camera, a mobile phone unit equippedwith a camera, a music player equipped with a camera, a game machineequipped with a camera, an electronic book reader equipped with acamera, and other apparatuses.

Aspects of the present invention can also be realized by a computer of asystem or apparatus (or devices such as a central processing unit (CPU)or a micro processing unit (MPU)) that reads out and executes a programrecorded on a memory device to perform the functions of theabove-described embodiments, and by a method, the steps of which areperformed by a computer of a system or apparatus by, for example,reading out and executing a program recorded on a memory device toperform the functions of the above-described embodiments. For thispurpose, the program is provided to the computer for example via anetwork or from a recording medium of various types serving as thememory device (e.g., computer-readable medium).

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments, and can include variousother embodiments without departing from the spirit or scope of thepresent invention. Further, the embodiment provided herein indicatesonly an example of an embodiment, and the features of the embodiment maybe arbitrarily combined with other embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all modifications, equivalent structures, and functions.

This application claims priority from Japanese Patent Application No.2011-101312 filed Apr. 28, 2011, which is hereby incorporated byreference herein in its entirety.

1. An imaging apparatus comprising an imaging unit; a generation unitconfigured to generate a multiple image combination result image bycombining a through-the-lens image captured by the imaging unit and atleast one already captured image; a focus adjustment unit configured toperform a focus adjustment by driving a focus lens; and a displaycontrol unit configured to perform control to display thethrough-the-lens image captured by the imaging unit on a display unitwhile sequentially updating the through-the-lens image when the focusadjustment unit is performing the focus adjustment, and display themultiple image combination result image generated by the generation uniton the display unit while sequentially updating the multiple imagecombination result image after the focus adjustment unit completes thefocus adjustment.
 2. The imaging apparatus according to claim 1, whereinthe display control unit performs control to display the multiple imagecombination result image generated by the generation unit on the displayunit while sequentially updating the multiple image combination resultimage before the focus adjustment unit performs the focus adjustment. 3.The imaging apparatus according to claim 1, wherein the focus adjustmentunit performs automatic focus processing.
 4. The imaging apparatusaccording to claim 1, wherein the focus adjustment unit performs amanual focus adjustment according to a user's operation.
 5. The imagingapparatus according to claim 1, wherein the display control unitperforms control to display the multiple image combination result imagegenerated by the generation unit on the display unit while sequentiallyupdating the multiple image combination result image after apredetermined time has passed from completion of the focus adjustment bythe focus adjustment unit.
 6. The imaging apparatus according to claim1, further comprising a shooting processing unit configured to shoot animage in a focus state adjusted by the focus adjustment unit to recordthe shot image in a recording medium according to a reception of ashooting instruction, when the display control unit displays themultiple image combination result image generated by the generation uniton the display unit while sequentially updating the multiple imagecombination result image after the focus adjustment unit completes thefocus adjustment.
 7. A method for controlling an imaging apparatusincluding an imaging unit, the method comprising: generating a multipleimage combination result image by combining a through-the-lens imagecaptured by the imaging unit and at least one already captured image, asa generation operation; adjusting a focus state by driving a focus lens,as a focus adjustment operation; performing control to display thethrough-the-lens image captured by the imaging unit on a display unitwhile sequentially updating the through-the-lens image when the focusstate is being adjusted by the focus adjustment operation, and displaythe multiple image combination result image generated by the generationoperation on the display unit while sequentially updating the multipleimage combination result image after the focus adjustment by the focusadjustment operation is completed, as a display control operation.
 8. Anon-transitory computer readable storage medium storing a programcapable of causing a computer to function as the respective units of theimaging apparatus according to claim 1.